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Chapter 1 


Statement of Intent and Findings 


The intent of this research is LES. document the 
feasibility of awarding federal construction contracts based 
on mean bid as an alternative to low bid. In this case, mean 
bid award is derived by summing the values of all bids 
received for an advertised federal construction contract and 
determining the average value. This value is the benchmark 
by which all bids are compared--the contractor who bids 
closest to this benchmark is awarded the contract. 

Using data from 55 Navy construction contracts collected 
from two Navy Engineering Field Divisions and one Navy 
Resident Officer in Charge of Construction office, a 
comparison of final contract price to the average of all bids 
received will be made. The premise for comparison is that 
mean bid award would ensure that a "sincere" bidder receives 
the contract, thus reducing the number of costly changes and 
improving project quality. 

Three methods were used in this study to determine the 
mean bid for each of the 55 contracts: (1) straight average 
of all bids received, (2) average of all bids received 
excluding the high and low bids, and (3) average of all bids 
received that fall within a range of 70% to 130% of the 
Government estimate. 


Each contract was reviewed independently to determine 





the actual award amount. Modifications other than customer 
requested changes were added to the award amount to determine 
the final price of the project. This final price was then 
compared to the mean bid. 

Using this comparison as the criteria of feasibility, 
federal construction contract award by mean bid is’ not 
supported within the specific context of thiS research 
because the majority of the contracts had final prices lower 
than the value of the average bid. However, as the Navy and 
the construction industry embrace the Total Quality 
Management philosophy, an alternate means of contract award 
such as mean bid should not be ruled out. Reasons for this 
conclusion will be developed in later chapters of this 


report. 





Chapter 2 


Introduction 


Although not the norm, it is not uncommon in federal 
construction contracting to encounter a bid for a project 
that may be considered to be "“insincere." An insincere bid 
may be defined as a very low bid offered for the purpose of 
obtaining the contract and seeking profit through excessive 
Changes. If such a bidder is responsive and responsible, the 
Government is obligated to award the contract and then face 
the consequences of such an award. 

Consider the following scenario: a contractor offers a 
bid on a Navy contract to rehab a Marine Corps Reserve Center 
that is remotely located from the Navy officer charged with 
administering the contract. The bid is lower than anti- 
Cipated, but because the contractor has been found to be 
responsive and responsible and does not acknowledge any 
mistakes in the bid, the contract iS awarded. Because the 
construction site is so remotely located, routine daily 
inspections by the Navy are not feasible and communications 
with the contractor are routinely only by telephone or 
letter. As the project progresses, the contractor initiates 
an excessive number of changes to the contract. The changes 
are priced extremely high in comparison to the initial bid 
for the project. Negotiations stall out and the Navy is 


forced to seek other means of accomplishing the work or to 





accept changes that are priced dramatically higher than 
anticipated. The end result is a project that is completed 
late, at a price that iS much higher than predicted, and 
lacks the desired quality. 

Such a scenario might justify an alternative type of 
contract award. If the contract were awarded to a contractor 
who bid closest to the mean or average of the bids received 
for that contract, this type of inflammatory situation might 
be avoided. The rationale is that the contractor who bid 
closest to the mean of all bids received, bid the contract 
"Sincerely" or accurately rather than as low as possible 
Simply to obtain the contract. Such a contract may avoid the 
excessive changes that an insincere bidder would invoke as a 
means of obtaining some profit or costs that were not init- 
ially bid, and may further avoid the resulting degradation of 
the relationship between the owner and the contractor. 

Other countries around the world have used _ this 
alternative form of contract award. A Presidential decree in 
the Philippines created a system where all bids are added to 
the client's estimate and then averaged. Any bidder whose 
price is less than 70% of the figure derived from this cal- 
culation is rejected. The client's estimate forms an upper 
limit. The successful bidder is the one closest to the 
benchmark derived from the averaging calculation. (Barrell 
1988). 


A similar system was established in 1974 in Italy for 





public sector construction. This system was developed due to 
the extremely high level of competition and concern that it 
was fueling claims and disputes. According to the Italian 
contractors' association, Associazione Nazionale Costruttori 
Edili (ANCE), the average bid system was effective in 
bringing prices up to a realistic level and in addressing 
pressures to make claims which had resulted from unrealist- 
ically low bidder prices. (Barrell 1988). 

As described by ANCE, bids are accepted within a range 
of -5% to -30% (variable by contract) of the estimate; 
outside that range the bids are rejected. Acceptable bids 
are averaged and the bidder nearest to the average is 
accepted. If there are two bids equally close to the 
estimate, one above and the other below, the higher bid is 
accepted. (This is apparently to further reduce the 
potential for claims that the unrealistically low bidder 
prices generated). (Barrell 1988). 

Involvement in the European Economic Community spelled 
the end of the system in 1978. However, according to ANCE, 
the Italian Government is currently attempting to have the 
system reinstated in a case before the EEC Court in Brussels. 
Also according to ANCE, there are other European countries 
interested in implementing the system. (Barrell 1988). 

An averaging system was also apparently in operation in 
Iran, prior to the fall of the Shah, by which the highest and 


lowest prices were rejected and the successful bidder chosen 





on the basis of closest proximity to the average of the 
remaining prices. A mean bid system has been tried in the 
private sector in the United States, but has not received 
widespread use. (Data regarding this award system were not 
available at the time of this study). Although unconfirmed, 
there have also apparently been instances where a mean system 
has been tried in Australia. (Barrell 1988). 

In other countries in which research was carried out, 
Organizations and individuals expressed almost’ universal 
skepticism or opposition to mean bidding systems. The point 
was made repeatedly that there is no justification for 
selection other than the lowest bidder, if pre-qualification 
Of bidders is used. The point was also made that public 
accountability makes it difficult, if not impossible, to move 
to such a system. It was questioned whether a mean system 
would result in a Significant increase in prices, as bidders 
adjusted their prices to try to find the mean point. It was 
also questioned whether the system would result in the 
development of efficient methods of construction and design 
alternatives, as the incentive to do so would be lost. 


(Barrell 1988). 





Chapter 3 


Background 


The data gathered for this research are from the United 
States Navy, one of the largest owners of facilities in the 
world. The Naval Facilities Engineering Command (NAVFAC) is 
responsible for the engineering, construction, operation and 


Maintenance of these facilities. 


3.1 Naval Facilities Engineering Command (NAVFAC) 

The mission of NAVFAC is to acquire and maintain all the 
Navy's shore facilities. Their $300 billion world-wide 
Physical plant includes operational facilities for 
submarines, surface ships, and aircraft, as well as all the 
base support for personnel, industrial activities, logistics, 
and communications. It logically follows that NAVFAC is 
engaged in virtually every type of construction, including 
industrial, commercial, heavy, and residential. Few owners 
can Claim that they have this diversity among all their 
project work. (Broaddus 1991). 

Although headquartered in Washington, DG, NAVFAC 
performs project conception, planning, procurement, 
construction and startup through seven regionally located 
Engineering Field Divisions (EFD's). Collectively, these 
EFD's are responsible for the Navy's facility construction 
and maintenance functions all over the world. Each EFD is 


asSigned a geographic region of responsibility. All major 





facility engineering and construction functions carried out 
at any of the Naval shore activities within a region fall 
under the jurisdiction of the assigned EFD. 

The planning and design functions of a construction 
project are performed either in-house at the EFD ofr 
contracted out according to the workload and capabilities of 
the EFD. The construction functions are contracted out to 
private contractors through a formalized procurement process. 
The construction contract is administered by the EFD's field 
offices or Resident Officer in Charge of Construction (ROICC) 
which are located at most Navy shore activities. The ROICC 
is responsible for all aspects of contract administration 
including site inspection, submittal review, modifications, 


contractor payment verification, startup and turnover. 


3.2 The Military Construction (MILCON) Program 

Virtually all major capital improvements for the Navy 
are done through the MILCON Program, though all major 
construction is not exclusively restricted to that program. 
For example, normal operation and maintenance funds may be 
used for major repairs where facilities are being essentially 
replaced in kind or where major damages require immediate 
attention. Notwithstanding this exception, the bulk of the 
work, as well as the most complex and demanding projects, are 
in the MILCON Program. 


The MILCON Program is funded annually by Congress as a 





separate and distinct appropriation. Every Navy project over 
$200,000 must be authorized and appropriated as a specific 
line item in an annual congressional budget. In brief, the 
process starts with requirements being identified at the 
local base level, or perhaps through addition of a new 


mission or weapons system requiring facilities at one or more 


selected locations. A project for construction is submitted 
up through the Operational Cchain-of-command, with EFD 
assistance, to the Chief of Naval Operations. Ce ees 


validated and prioritized sufficiently, it will earn a spot 
within a five-year defense program. 

When a project is within three years of its projected 
budget year, the planning process starts to further define 
the scope in preparation for design authorization. In some 
cases, "front end" planning studies are either conducted in- 
house or by architect-engineer (A-E) firms to further define 
the project. At approximately two years prior to the project 
funding year, the design of the project is officially 
authorized. At this time, an A-E firm is selected on the 
basis of qualifications, and a contract is negotiated to 
complete all plans and specifications for the project. 

Once a project has reached the 35 percent design 
completion stage, it is ready to go into the Department of 
Defense and Presidential budgets for submission to Congress 
for hearings and eventual authorization and appropriation. 


Of course, many projects do not survive the entire budget 





process. A project may be deferred to a later year by 
Defense Department budget analysts or eliminated completely 
by lack of support in one of four Congressional committees. 
Also, once the House and Senate Armed Services and 
Appropriations Committees joint recommendation becomes law, 
the authorization and appropriations bills come with a 
variety of "strings attached." These normally relate to the 
cost growth allowable and the maximum time in which the 
project must be started; however, there are no significant 
congressional constraints on project completion. 

Even though the design completion reaches 35 percent and 
a project may be included in the President's budget, design 
activity normally continues while the budget considerations 
are under way. Frequently, by the time the Congressional 
budget is approved and funds apportioned, the design is 
complete and ready for advertising, bidding and award. As 
with any political process, projects are dropped and the 
authorization and appropriation bills are often late. 
Ideally, the Military Construction legislation is scheduled 
for passage prior to October lst each year since that date 
serves as the start of the federal fiscal year. Once a bill 
becomes law, it is legal for the construction contract to be 


awarded for all or parts of the project. (Broaddus 1991). 


3.3 The Minor Construction Process 


Commanding Officers of Navy shore activities are 
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authorized to spend up to $200,000 for individual minor 
construction and repair contracts without Congressional 
approval. The process through which these projects’ are 
executed is much simpler than that of the MILCON Program. 
Specifically, the Commanding Officer decides which projects 
have priority, and those projects are funded up to the limit 
the activity's annual minor construction budget. The 
activity's local Public Works Center or department (depending 
on the size and location of the Navy shore activity) develops 
the conceptual planning for the project. Detailed 
engineering and design is performed either in-house or 
contracted out. After plans and specifications are 
completed, the project is ready for advertising, bidding and 
award. The EFD does not generally get involved in these 
station contracts; however it is available for guidance, if 


necessary. 


3.4 Advertising, Bidding and Contract Award 

Whether funded through the MILCON Program or funded as a 
station contract, once the plans and specifications for a 
project are completed and approved, the project is ready for 
advertising, bidding and award. These functions are 
generally carried out by the local contracts office at the 
Navy shore activity where the project is to be constructed. 

In brief, the project is advertised for thirty days in 


the Commerce Business Daily, a monthly publication that lists 


Wel 





federal construction projects open for bidding. Private 
contractors submit sealed bids to the local contracts office 
through which the contract will be awarded. At the specified 
date and time, all sealed bids are opened and reviewed. The 
lowest bidder that is found to be responSive and responsible 
is awarded the construction contract. The ROICC is notified 
of the contract award and establishes a date and time to 
conduct a pre-construction briefing with the contractor and 
user of the facility. 

The ROICC administers the construction contract through 
completion of the facility, ensures contract compliance and 


conducts turnover to the user. 
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Chapter 4 


Methodology 


In 1988 the Department of Defense and the Navy began its 
drive to embrace a new strategy of leadership called Total 
Quality Management (TQM) or Total Quality Leadership (TQL). 
The key elements to this management philosophy, are: 

-- There must be continuous improvement in all aspects 
of an organization. 

-- This improvement is the responsibility of all members 
of the organization. 

-- Data, statistical methods, and careful analysis are 
essential to gain the improvement. 

-- Ultimately, the customer determines the quality of 
both products and services. 

The focus of TQM is on work processes--those repetitive 
steps that start with an input from a supplier and end with a 
product or service delivered to a customer. From this macro 
perspective, the suppliers and the customers are outside the 
Organization. But within the macroprocesses are dozens and 


sometimes thousands of microprocesses that exist solely 


inside the organization. Both the supplier and the customer 
reside in other parts of the same organization. (Mumford 
7991). 


Dr. W. Edwards Deming, internationally renowned for his 


expertise in TQM, developed his Fourteen Points to guide 
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organizations as they implement total quality programs for 
the continuous improvement of operations, service quality and 
productivity. Dr. Deming's fourth point is "end the practice 
of awarding business on the basis of price tag alone" which 
usually leads to low service quality and predictable cost 
overruns. His recommendation is to shift the emphasis away 
from seeking the lowest bidder and to seek and initiate 
alternative means of procurement. Mean bid contract award 


potentially represents a vehicle for adopting this change. 


4.1 What to Measure 

To conduct this study, it is necessary to compare the 
final price of a construction contract awarded by low bid to 
the average of all bids received for that project. This 
comparison will provide a means to evaluate the feasibility 
of awarding construction contracts by mean bid versus’ low 
bid. For the purposes of this research, the final price of a 
contract is the award price plus all additive and deductive 
changes, excluding customer requested changes. Customer 
requested changes are not considered because it is assumed 
that they are completely beyond the control of the contractor 
and represent changes in scope from the project as originally 
advertised and bid. 

The rationale behind the comparison is that contracts 
awarded by low bid are fertile ground for the contractor who 


submitted an "insincere" bid to seek changes and drive up 
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costs. Conversely, a mean bid system should award contracts 
only to "sincere" bidders, lessening the chances for 
excessive changes. By comparing the final price to the 
average of all bids received, it is possible to see if it is 


economically feasible to award by mean bid. 


4.2 How to Measure 

To conduct the comparison of final price to the average 
of all bids received, contract data were gathered from 55 
Navy construction contracts. To ensure diversity, the data 
were gathered from a variety of sources: 30 contracts from 
the ROICC office at Naval Air Station Memphis, TN; 1/7 
contracts from the EFD located in Charleston, SC (Southern 
Division, Naval Facilities Engineering Command); and 8 
contracts from the EFD located in Philadelphia, PA (Northern 
Division, Naval Facilities Engineering Command). The value 
of the contracts varied in price from $25,000 to $15 million. 
Both the Military Construction (MILCON) Process and the Minor 
Construction Process are represented. 

For each contract, the scale of offers indicating all 
bids received was collected, the Government estimate was 
determined, the award price (low bid) was identified, all 
change order data were carefully reviewed, and the final 
price was calculated. One shortcoming of the data collection 
is that the final prices of the contracts don't include costs 


incurred due to claims because such data were not available. 
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4.3 How to Analyze 

Three methods of determining the mean bid were used in 
the analysis of the data. The first method was to simply 
determine the arithmetic average of all bids received for 
each contract. This method reflects the average of all bids 
regardless of the magnitudes of the bids. 

The second method was to determine the arithmetic 
average of all bids excluding the high and low bids. This 
method refines the mean or benchmark value by excluding any 
stray bids that may contain errors thus pulling the average 
away from a realistic value. 

The third method was to determine the arithmetic average 
of all bids falling within a range of 70% to 130% of the 
Government estimate. This method ensures that all bids are 
in line with the predicted cost of the contract. The 
shortcoming of this approach is that it assumes’ the 


Government estimate is accurate which is not always the case. 


4.4 How to Prove Worth 

Proving the worth of the analysis of the research is 

based on the following premise: 

(1) If the final price of the contract is HIGHER than 
the average of the bids received, an award based on 
mean bid IS supported. 

(The assumption is that the mean bid award amount will be 


closer in value to the final price due to less changes, fewer 
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claims, and higher quality. i.e. BID = FINAL PRICE). 

(2) If the final price of the contract is LOWER than 
the average of the bids received, an award based on 
mean bid is NOT supported. 

(The assumption is that even with changes, claims, etc., 
awarding the contract by low bid results in a final price 
that is lower than the average bid). 

A strict interpretation of this method of analysis would 
mean that if the final prices of the contracts are higher 
than the average bids, it would be wise to award by mean bid. 
Conversely, if the final prices of the contracts are lower 
than the average bid, it would not be wise to award by such a 
method. However, as will be discussed in the conclusion of 
this report, such strict interpretation of these results may 
preclude the use of a potentially useful means of contract 


award. 


4.5 Possible Disadvantages 

It can be argued that certain inherent disadvantages 
exist in the mean bid system. Full consideration must be 
given to these disadvantages prior to the implementation of 
such a system. 

Disadvantage #1: Mean bid award may remove contractor 
incentive to develop more efficient construction methods. If 
a contractor is trying to develop a bid which will be awarded 


based on the average of all bids received, that contractor 
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would not be inclined to submit a bid based on an alternate, 
more efficient means of constructing the project. Rather 
than look for means to lower the bid which could potentially 
Save the owner money, the contractor may only bid the project 
strictly as designed. Also, the contractor's incentive to 
make more profit by using a more efficient technique may be 
jeopardized. 

Disadvantage #2: Mean bid award may simply start the 
project from a higher plateau from which to add costs. The 
phrase: "There's no such thing as a perfect set of plans and 
specs," is commonly heard in the construction industry. 
Consequently, even if the contractor bids sincerely on the 
project, it is likely that there will be some modifications 
to the contract. If the contract is awarded on mean bid, 
then the final price of the project may simply grow to higher 
proportions than if the contract were awarded by low bid. 
This fact alone could make the use of the mean bid award 
system in the public sector very difficult to justify. 

Disadvantage #3: The mean bid award system may induce 
bidder collusion. It is possible that a group of contractors 
may collude with each other to determine where the average 
bid will lie. This type of collusion would probably be very 


rare. 
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Chapter 5 


Presentation and Analysis of Data 


The comparison of the final price of a construction 
contract to the average of all the bids received represents 
the heart of this research. The 55 Navy construction 
contracts studied represent a wide variety of contracts and 
Should therefore be an adequate cross-section of federal 
construction contracting. As described in Chapter 4, the 
comparison was made using three approaches: (1) the Straight 
Average Method, (2) the High/Low Bids Excluded Method, and 
(3) the Government Estimate Range Method. The results of 
these approaches will be presented and analyzed below. 
Appendix A contains 55 data tables showing the pertinent data 
for each of the contracts and the tabular results of each of 


the three methods of analysis. 


5.1 The Straight Average Method 

This approach involved determining the average bid value 
based on all bids received regardless of magnitude of the 
bids. Of the 55 contracts studied, only eight contracts 
support the mean bid technique. This is graphically 
portrayed in the following three graphs. Figure 5.1 shows 
those contracts whose bids ranged in value up to $100,000; 
Figure 5.2 shows those contracts whose bids ranged in value 


from $100,000 to $1,000,000; and Figure 5.3 shows those 
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contracts whose bids ranged in value from $1,000,000 to 
peo, 000,000. 

The X-axis of the graphs represents the average bid and 
the Y-axis represents contract final price. The solid 
diagonal line through the graph represents the function, Y=xX, 
Or Final Price is equal to Average Bid. All data points 
above the line represent contracts whose final price is 
higher than the average bid (mean bid system IS supported). 
All data points below the line represent contracts whose 
final price is lower than the average bid (mean bid system 
NOT supported). This graphing technique iS consistent 


through all three methods of analysis. 
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Figure 5.1 
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Average Bid vs. Final Price 
(Bids: $100,000 to $1,000,000) 
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Figure 5.2 


Average Bid vs. Final Price 
(Bids: $1,000,000 to $15,000,000) 
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5.2 The High/Low Bids Excluded Method 

This approach involved determining the average bid value 
excluding the highest and lowest bids. This refines the 
average in case there is a stray bid which is’ uncharac- 
teristically high or low. Of the 55 contracts studied, only 
eight contracts support the mean bid technique. This is 
graphically portrayed in the following three graphs. Figure 
5.4 shows those contracts whose bids ranged in value up to 
$100,000; Figure 5.5 shows those contracts whose bids ranged 
in value from $100,000 to $1,000,000; and Figure 5.6 shows 


those contracts whose bids ranged in value from $1,000,000 to 


$15,000,000. 
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Figure 5.4 
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Average Bid vs. Final Price 
(Bids: $100,000 to $1,000,000) 
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Figure 5.5 
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5.3 The Government Estimate Range Method 

This approach involved determining the average of those 
bids within a range of 70% to 130% of the Government 
estimate. This further refines the mean to a predictable 
value. Of the 55 contracts studied, eleven contracts support 
the mean bid technique. This is graphically portrayed in the 
following three graphs. Figure 5.7 shows those contracts 
whose bids ranged in value up to $100,000; Figure 5.8 shows 
those contracts whose bids ranged in value from $100,000 to 
$1,000,000; and Figure 5.9 shows those contracts whose bids 


ranged in value from $1,000,000 to $15,000,000. 
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Figure 5.7 
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Average Bid vs. Final Price 
(Bids: $100,000 to $1,000,000) 
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Figure 5.8 
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Figure 5.9 
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5.4 Variation of Bids 

After making the comparison and graphically showing how 
many contracts strictly support the premise of this research, 
it 1S important to determine how close in value the final 
prices of the contracts were to the average bids. Figure 
5.10 portrays how the contracts supporting the mean bid 
system broke out in relation to their proximity of the final 
price to the mean bid. All three different methods of 
determining the mean bid value had similar results: the 
majority of the contracts' final price was within 1% to 20% 
of the average bid. Only five contracts fell outside this 


range. 


Variation of Average Bid to Final Price 
Contracts: Final Price > Average Bid 
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Figure 5.10 
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Similar results were found of those contracts that don't 
support the mean bid system: approximately one half of the 
contracts' final price was within 1% to 20% of the average 
bid. The majority of the remaining contracts had a final 
price between 20% and 60% of the average bid. Figure 5.11 


indicates these results. 


Variation of Average Bid to Final Price 
Contracts: Final Price < Average Bid 
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Chapter 6 


Conclusions 


The conclusions that can be drawn from this research are 
twofold. On the one hand, there are those conclusions which 
can be made based on a strict interpretation of the data. On 
the other hand, there are conclusions which may be drawn from 
a less rigorous interpretation of the data. 

Strictly speaking, the results of the analysis of the 
data presented do not support a mean bid system. Of the 55 
contracts, only eight contracts Support a mean bid system 
under the Straight Average Method; only eight contracts 
support a mean bid system under the High/Low Bids Excluded 
Method; and eleven contracts support a mean bid system under 
the Government Estimate Range Method. Of those contracts in 
support, the majority of them had final prices within 20% of 
the average bid. This is not a tremendous’ savings 
considering that there will still be some change orders to 
add to the final price even under a mean bid system. 

A less rigorous interpretation of the data may yield 
more support to a mean bid system. This research effort did 
not address some of the follow-on costs to the construction 
phase: claims and dispute resolution costs (if any), repair 
and warranty costs, and operations and maintenance costs. 
The final price of the 55 contracts was based only on the 


award price plus additive and deductive change orders 
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(excluding customer requested changes and scope changes). A 
more accurate final price to compare to the average bid would 
include some of these additional follow-on costs. Conducting 
a Similar study with these data available to determine the 
final price could produce results in favor of a mean bid 
system. 

The Total Quality Management philosophy which the 
Department of Defense and the Navy are diligently working to 
adopt stresses that business contracts should not be awarded 
on the basis of lowest price. In the public sector, this is 
a difficult idea to implement. As custodians of public 
funds, public officials and military leaders have the 
responsibility of obtaining the highest quality product at 
the lowest possible price--in other words, not wasting the 
public's money. However, the fundamental goal of TOM is to 
provide a better product that completely satisfies the 
customer. This means not only meeting initial costs goals, 
but also providing a facility that is easy to operate and 
maintain. Construction projects are complex by their nature; 
they are expensive and time consuming. Adopting TQM isa 
means of ensuring that projects produce high quality 
facilities that satisfy the customer. Using an award system 
Similar to mean bid can put the federal Government one step 


closer to successfully implementing TOM. 
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Chapter 7 


Recommendations for Further Study 


This research effort could not address some of the 
follow-on costs to the construction phase of a project: 
claims and dispute resolution costs, repair and warranty 
costs, and operations and maintenance costs. Such data are 
critical to conduct the large scope research necessary to 
make the decision to implement a mean bid award system. 

The current low bid award system has certain inherent 
shortcomings that do not agree with Total Quality Management 
Philosophy. But before this system can be retired, an award 
system that fully meets the needs of the public and the 
federal Government must be determined. Whether such a system 
is based on mean bid, negotiated procurement, etc., is yet to 
be seen. 

It is recommended that a large scale research effort 
that has access to all follow-on costs be executed in order 
to help determine an alternative to low bid award. However, 
it may take years to conduct such a study due to the data's 


complexity, especially if it's tied up in litigation. 
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